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COMPUTER CONTROLLED INTERACTIVE TOUCH DISPLAY PAD 
WITH TRANSPARENT FULL CHARACTER KEYBOARD OVERLAYING 
DISPLAYED TEXT AND GRAPHICS 

Technical Field 
5 The present invention relates to user interactive 

computer supported display technology and particularly to 
such user interactive systems and methods related to 
touch screen technology. 

Background of Related Art 

10 In recent years, convergence of the data processing 

industry with the consumer electronics and communications 
industries has accelerated extensive consumer and 
business involvement in computer driven technologies. As 
a result of these changes, all aspects of work in 

15 business and technology requires human/computer 

interfaces. There is a need to make computer directed 
activities accessible to a substantial portion of people 
who up to a few years ago, were computer illiterate, or 
at best, computer indifferent. In order for the 

20 extensive computer supported market places to continue 
and be commercially productive, it will be necessary for 
a large segment of computer indifferent workers and 
consumers to be involved in computer interfaces. Thus, 
the challenge of technology is to create interfaces to 

25 computers that are as close to the real world as 
possible. 

In this connection, the mouse, which has been the 
primary input device to computers for a generation, is 
still considered to be a physically awkward input device. 
30 Consequently, there has been considerable effort in the 
development of the touch screen or touch panel. It is 
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easy to use because it allows the user to point directly 
to the display screen with his finger or a pen or stylus 
to make selections that result in graphic or alphanumeric 
changes on the display screen. The touch panel, in 
5 various forms, has been in use for several years. 
Several different technologies have been involved in 
touch panels. Original touch panels used a series of 
infrared LEDs and light sensors, such as photodiodes, to 
provide low resolution panels of up to 50 resolvable 

10 positions. The LEDs and sensors form a grid of invisible 
light beams that the finger breaks to, thus, indicate its 
position. The capacitively coupled touch panels were 
able to develop a resolution of about 100 resolvable 
positions. Higher resolution touch screens have been 

15 developed using a variety of technologies from sound 
waves reflected off fingers to conductive/resistive 
layers separated by insulative material broken down by 
touch . 

Recent developments, which have dramatically 
20 improved the clarity and resolution of liquid crystal 
displays, as well as the improved resolution and 
responsiveness in touch panels overlaid on such liquid 
crystal displays, have renewed the hopes among students, 
academics, researchers and literary developers for an 
25 effective paperless notebook. Such a notebook should be 
one which is light and portable; can store up to several 
books, sequences of pages of which may be read via the 
display screen; enables the entry of cursive drawings, 
graphics and script, e.g. class or scientific notes and 
30 annotations respective to the text and images; as well as 
the entry of typed text. 

The latter has presented a problem because for 
greatest convenience it requires a full keyboard. This 
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has led to awkward notebook structures with a separate 
swing-out touch keyboard not unlike laptop PCs. Also, 
some touchpads have a separate keyboard apart from the 
displayed text and graphics document for typed text 
5 entries. Other simpler display notebooks have some 
minimal pecking in of alphanumeric characters but not 
from a full keyboard array. 

Summary of the Present Invention 

The present invention provides a solution to the 

10 above-mentioned shortcomings of touch screen display 
notebooks by eliminating the hinged swing-out keyboard 
while having a full keyboard available for the entry of 
typed text, as well as a simultaneous display of the 
text /graphics document being read or modified. To 

15 achieve such results, the present invention provides a 
portable computer controlled user interactive touch 
responsive read/write display pad comprising the 
combination of a display screen displaying text and 
graphics, a transparent touch sensitive pad covering said 

20 display screen, means responsive to cursive drawing or 
characters touch input for displaying such drawing or 
characters on said display screen, means for displaying a 
full character keyboard on said display screen and means 
responsive to touch inputs to characters on said keyboard 

25 for displaying said touch keyboard inputs as text entries 
within said displayed text and graphics. 

This full character keyboard is preferably 
superimposed upon and transparent to said displayed text 
and graphics in the displayed document. There is further 

30 included means for selectively rendering either one of 
said means responsive to cursive drawing or said means 
responsive to inputs to touch keyboard non-interactive 
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while the other of said means remains interactive to 
thereby permit the unimpeded functioning of the means 
remaining interactive. In accordance with an aspect of 
the invention, there are means for varying the 
5 transparentness of said superimposed keyboard whereby 
said keyboard is less transparent when said keyboard is 
interactive and more transparent when said keyboard is 
non-interactive . 

There is also provided means responsive to an input 
10 to a key in said keyboard for rendering interactive said 
means responsive to keyboard inputs and for rendering 
non-interactive said means responsive to cursive drawing. 

Brief Description of the Drawings 

The present invention will be better understood and 
15 its numerous objects and advantages will become more 

apparent to those skilled in the art by reference to the 
following drawings, in conjunction with the accompanying 
specification, in which: 

Fig. 1 is a block diagram of a data processing 
20 system including a central processing unit, an LCD 

display and overlaid touch panel display that is capable 
of implementing the present invention; 

Fig. 2 is a diagrammatic general three-dimensional 
view of the touch sensitive notebook of the present 
25 invention; 

Fig. 3 is a diagrammatic view of the notebook 
display screen of the present invention having a page of 
a book or document with text and graphics; 

Fig. 4 is the diagrammatic view of the displayed 
30 page of Fig. 3 during editing or cursive data entry via a 
stylus; in this mode the superimposed keyboard has a high 
transparentness ; 
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Fig. 5 is the diagrammatic view of the displayed 
page of Fig* 3 during editing or text data entry via the 
keyboard; in this mode the superimposed keyboard has a 
relatively low transparentness ; 
5 Fig. 6 is a flowchart of the basic elements of the 

program in a computer controlled display system of this 
invention for coordinating the input to the touch display 
with the transparentness of the keyboard; and 

Fig. 7 is a flowchart of the steps involved in an 
10 illustrative run of data entry or document editing within 
the process set up in Fig. 6. 

Detailed Description of the Preferred Embodiment 

Referring to Fig. 1, a data processing system is 
shown which may function as the computer controlled 

15 display terminal with the touch pad data entry display 
coupled to the primary display in accordance with the 
present invention. The computer controlled primary LCD 
display may be a LCD display personal computer unit made 
up of a central processing unit (CPU) 10, such as one of 

20 the PC processors available from International Business 
Machines Corporation (IBM), Dell Corp. or Compaq Corp., 
which is provided and interconnected to various other 
components by system bus 12. An operating system 41 runs 
on CPU 10 and provides control and is used to coordinate 

25 the functions of the various components of Fig. 1, 

including the touch screen 21 LCD display 23 pad which 
will hereinafter be described in greater detail. 
Operating system 41 may be one of the commercially 
available operating systems such as Microsoft Windows95™ 

30 or WindowsNT"™, as well as UNIX or IBM's AIX operating 
systems. A program for simultaneously displaying the 
images of the primary display document or book on the 
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touch pad 21 display 23 and for applying the superimposed 
transparent keyboard image to the LCD display, 
application 40, to be subsequently described, runs in 
conjunction with operating system 41 and provides output 
5 calls to the operating system 41 which implement the 

various functions to be performed by the application 40. 
A read only memory (ROM) 16 is connected to CPU 10 via 
bus 12 and includes the Basic Input/Output System (BIOS) 
that controls the basic computer functions. Random 

10 Access Memory (RAM) 14, I/O adapter 18 and communications 
adapter 34 are also interconnected to system bus 12. It 
should be noted that software components, including 
operating system 41 and application 40, are loaded into 
RAM 14, which is the computer system's main memory when 

15 the system is in operation. Otherwise, when the system 
is dormant, most of the software, including operating 
system 40 and applications 41, are stored in disk storage 
device 20. I/O adapter 18 may be a Small Computer System 
Interface (SCSI) adapter that communicates with the disk 

20 storage device 20, i.e. a hard drive. Communications 
adapter 34 interconnects bus 12 with an outside network 
enabling the data processing system to communicate with 
other such systems over a Local Area Network (LAN) or 
Wide Area Network (WAN), which includes, of course, the 

25 Internet. Thus, the system of the present invention may 
be used with screens and pages received over the 
Internet. I/O devices, such as keyboard 24 and mouse 32, 
are also connected to system bus 12 via user interface 
adapter 22. It is through such input devices that the 

30 user may make conventional data entries. 

With respect to the display pad input devices used 
in the invention, any conventional touch screen display 
may be used. Typically, in Fig. 1, there is an LCD 
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display having surface 23 upon which the visual output 
from the computer is generated via pad display adapter 
30. A touch sensitive display panel 21 is superimposed 
upon display surface 23. This touch screen, which is 
5 about 1/4" to 1/2" from surface 23, is responsive to a 
touch stimulus, i.e. stylus 27, applied by the user to 
make the graphic, cursive and other entries to be 
subsequently described. The touch screen 21 resolution 
is determined by digitizing circuitry (not shown) in a 

10 touch screen adapter 25 to form a two-dimensional array 
of discrete coordinate points, A touch stimulus applied 
by stylus 27 or by the user's fingers in the superimposed 
keyboard to any of the coordinate points is detected by a 
sensor array (not shown) in the touch panel 21. The 

15 sensor array generates an analog signal responsive to the 
force imparted to the touch screen. The stimulus may 
also be imparted proximate to the touch screen. The 
signal is digitized by a sampling A to D convertor 
circuit (not shown) in touch panel 21 to produce an input 

20 data value. This data value, together with the 

coordinates to which it relates, are transmitted from 
touch panel 21 to touch pad display adapter 25. The 
input data value corresponding to each set of coordinates 
is conventionally refreshed by the A to D converter 

25 circuit about 60 times a second. The touch pad adapter 
25 connected to the bus architecture 12 passes each set 
of coordinates to the bus architecture 12 to the CPU 10 
and operating system 41 wherein this input is applied to 
the LCD display 23 via LCD display adapter 30. 

30 The touch panels or screens 21 may use any of the 

standard technologies discussed above. In the present 
embodiment, higher resolution panels using 
resistive/conductive composites should provide best 
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results. Such structures use two slightly separated 
layers of transparent material, one coated with a thin 
layer of conductive material and the other with resistive 
material. The pressure of the stylus forces the layers 
5 to touch and the voltage drop across the resistive 
substrate is measured and used to determine the 
coordinates of the touched positions. There are many 
such conductive/resistive touch screen displays on the 
market that may be used in the implementation of the 

10 present invention, such as the IBM 2489 Model 600 and PGI 
Super Nightingale. 

A three-dimensional view of the housing for the 
notebook display of the present invention is shown in 
Fig. 2. The notebook housing 45 includes touch sensitive 

15 display screen 46 showing a page of a book or document 
with printed text 47 and graphics 48. There is a 
superimposed displayed keyboard 50 and some controls 51 
for the notebook. 

There will now be described a simple illustration of 

20 the present invention with respect to the display screens 
of Figs. 3 through 5. When the screen images are 
described, it will be understood that these may be 
rendered by storing image and text creation programs, 
such as those in any conventional window operating system 

25 in the RAM 14 of the system of Fig. 1. The operating 
system is diagrammatical ly shown in Fig. 1 as operating 
system 41. The display screens of Figs. 3 through 5 are 
presented to the viewer on LCD display 23 of Fig. 1. 

Fig. 3 shows an illustrative display screen during 

30 a document or book reading session. The display may be 
scrolled or paged conventionally. The screen 46 contains 
text 47 and graphics 48. If the user decides to edit, 
annotate or take class notes in his notebook, he may 
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switch into an edit mode as shown in Fig. 4 where he may 
use pressure stylus 51 in the conventional manner to add 
cursive notes 52 or cursive drawing or graphics 53. 
During this edit session, a full keyboard 50 will appear. 
5 The keyboard is transparent so that the user may still be 
able to see the underlying text and graphics. Since the 
editing in Fig. 4 is being done in the cursive mode, the 
keyboard 50 has a high transparentness , i.e. it is a 
"ghost" image to indicate its position and presence. The 

10 operating system graphics may be set up so that the 

keyboard entry edit mode for the entry of typed text is 
switched to when the user hits any key in the keyboard. 
Alternatively, one of the keys in the keyboard may be set 
up to be the switch key. In the present example, "y" key 

15 56 performs this role, i.e. the user must hit this key to 
switch to the keyboard entry mode shown is Fig. 5. In 
this keyboard entry mode, the keyboard 50 becomes less 
transparent and more clearly defined for ease of use. 
The keyboard is then used for entry of text 55 or for 

20 keystroke editing 55. While stylus 51 does not function 
during the keyboard entry mode of Fig. 5, it is included 
in the figure just for illustrative comparison purposes. 
In carrying out the functions of Figs. 4 and 5, the touch 
responsive display pad may be set up to respond 

25 conventionally to the stylus pressure. In the keyboard 
mode of Fig. 5, pressure response programs may be set up 
so that pressure applied at the positions of each of the 
respective keys is translated to an image of the 
character on the screen at the point of editing. 

30 Now, with reference to Figs. 6 and 7, we will 

describe a process implemented by the present invention 
in conjunction with the flowcharts of these figures. 
Fig. 6 is a flowchart showing the development of a 
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process according to the present invention for a touch 
sensitive notebook with text entry via a superimposed 
keyboard. A routine is set up for displaying documents 
or books with graphics and text on a touch display, step 
5 61. A routine is set up for entering cursive drawing and 
script via a stylus into the displayed document, step 62. 
A routine is provided for displaying a full character 
keyboard on the touch display superimposed upon at least 
a portion of the displayed text and graphics, step 63. 

10 Means are provided for activating the transparent full 
keyboard so that the touch sensitivity to the stylus is 
rendered inactive when the keyboard is activated, step 
64. A routine is provided for translating touches at the 
key positions into corresponding text entered into the 

15 displayed book or document, step 65. A routine is 

provided for deactivating the keyboard and activating the 
touch sensitivity to the stylus for cursive entries, step 
66. A routine is also provided for increasing the 
transparentness of the keyboard when the keyboard is 

20 deactivated, step 67. 

The illustrative running of the process will now be 
described with respect to Fig. 7. First, step 70, the 
text/graphics document or book page is displayed. The 
user selects the edit mode and the transparent keyboard 

25 is displayed superimposed over the text/graphics, step 
71. A determination is then made as to whether the user 
has selected the keyboard text-entry mode of editing, 
step 72. If Yes, the keyboard is made less transparent, 
i.e. more clearly displayed, step 73, and desired text 

30 entries are made in the document via the touch keyboard, 
step 74. If the determination in step 72 is No, the 
keyboard mode has not been selected, then step 45, the 
keyboard is made more transparent, i.e. a "ghost" 
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outline, step 75, and stylus script and drawing entries 
are made into the document, step 76. Upon the completion 
of either a text, step 74, or a stylus entry, step 76, 
function, a determination may conveniently be made as to 
5 whether the session is at an end, step 77. If Yes, the 
session is exited. If No, the session is returned to 
step 72 where a further determination is made as to 
whether the user has selected the keyboard text-entry 
mode of editing and the process is repeated with respect 

10 to the current or new page. 

Although certain preferred embodiments have been 
shown and described, it will be understood that many 
changes and modifications may be made therein without 
departing from the scope and intent of the appended 

15 claims. 



